Measuring Social Solidarity During Crisis: The Role of Design Choices

Building on our previous work, we assess how social solidarity towards migrants and refugees has changed before and after the onset of the COVID-19 pandemic, by collecting and analyzing a large, novel, and longitudinal dataset of migration-related tweets. To this end, we first annotate above 2000 tweets for (anti-) solidarity expressions towards immigrants, utilizing two annotation approaches (experts vs. crowds). On these annotations, we train a BERT model with multiple data augmentation strategies, which performs close to the human upper bound. We use this high-quality model to automatically label over 240 000 tweets between September 2019 and June 2021. We then assess the automatically labeled data for how statements related to migrant (anti-)solidarity developed over time, before and during the COVID-19 crisis. Our findings show that migrant solidarity became increasingly salient and contested during the early stages of the pandemic but declined in importance since late 2020, with tweet numbers falling slightly below pre- pandemic levels in summer 2021. During the same period, the share of anti-solidarity tweets increased in a sub- sample of COVID-19-related tweets. These findings highlight the importance of long-term observation, pre- and post-crisis comparison, and sampling in research interested in crisis related effects. As one of our main contributions, we outline potential pitfalls of an analysis of social solidarity trends: for example, the ratio of solidarity and anti-solidarity statements depends on the sampling design, i.e., tweet language, Twitter-user accounts' national identification (country known or unknown) and selection of relevant tweets. In our sample, the share of anti-solidarity tweets is higher in native (German) language tweets and among 'anonymous' Twitter users writing in German compared to English-language tweets of users located in Germany. © 2020 Tsinghua University Press.

Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset

We introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic. To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs. crowds). We use these annotations to train a BERT model with multiple data augmentation strategies. Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58% macro-F1 to almost 85%. We use this highquality model to automatically label over 270k tweets between September 2019 and December 2020. We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis. Our results show that solidarity became increasingly salient and contested during the crisis. While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.